Information processing system and method, information processing apparatus, image-capturing device and method, recording medium, and program

ABSTRACT

A feature extracting unit obtains sensor data from a plurality of sensors to calculate each feature. When an event determining unit determines the occurrence of an event based on each feature, a display data constructor generates remote-controller display data for displaying the event, and controls a remote-controller display device to display the remote-controller display data. When a user decision is input from a user input IF based on this display, a control unit controls the sensors to be turned ON or OFF. When an infrared sensor detects an abnormality, a microwave sensor whose power consumption is small after the infrared sensor is turned ON. When the microwave sensor detects an abnormality, a video camera and a microphone are turned ON, and the microwave sensor is turned OFF. A communication unit wirelessly transmits an image signal captured by the video camera and an audio signal processed by the microphone. Then, if the infrared sensor does not detect an abnormality, the video camera and the microphone are turned OFF. With this arrangement, power consumption can be suppressed. The present invention is applied to, for example, a security system, for example, for monitoring outside a vehicle by a video camera disposed in the vehicle when the vehicle is parked.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of application Ser. No.12/642,311, filed on Dec. 18, 2009, which is a division of applicationSer. No. 11/763,664, filed on Jun. 15, 2007, now U.S. Pat. No.7,840,284, which is a division of application Ser. No. 10/681,242, filedon Oct. 9, 2003, now U.S. Pat. No. 7,602,413, the entire contents ofeach of which are incorporated herein by reference. The presentapplication also claims priority to Japanese Patent App. No.2002-303795, filed Oct. 18, 2002, Japanese Patent App. No. 2003-005316,filed Jan. 14, 2003, and Japanese Patent App. No. 2003-013686, filedJan. 22, 2003.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to information processing systems andmethods, information processing apparatuses, recording media, andprograms. More particularly, the invention relates to an informationprocessing system and method, an information processing apparatus, arecording medium, and a program in which events can be flexibly detectedand reported so as to inhibit power consumption. The invention alsopertains to an image-capturing device and method, a recording medium,and a program in which the situation outside a vehicle can be monitoredby a back monitor camera disposed in the vehicle while the vehicle isbeing parked.

2. Description of the Related Art

Hitherto, in home security systems, a method for obtaining sensorinformation by viewing a monitor television (TV) that displays monitorimages sent from an image-capturing device has been suggested (forexample, Japanese Unexamined Patent Application Publication No.08-124078).

Another method for detecting humans entering a monitored area bydetermining the presence or absence of human bodies or traces by using amonitor device formed as a combination of an infrared sensor and animage sensor has been suggested (for example, Japanese Unexamined PatentApplication Publication No. 2000-339554).

In the inventions disclosed in the above publications, principally,specific places or specific abnormalities (events) are detected.

In the inventions disclosed in the above publications, however,adjustments required for installing the sensors (the image-capturingdevice and the monitor device) are troublesome, and once they areinstalled, it is difficult to move them to another place.

If power is supplied to a security camera by a cable from indoors, theinstallation place of the security camera is also restricted.Accordingly, power is supplied to the individual elements of thesecurity camera from batteries, thereby increasing the flexibility toinstall the camera.

In this case, however, the life of the batteries is short, and when thebatteries have run out, the security camera can no longer captureimages. A user is thus required to regularly charge or replace thebatteries, which is troublesome, and the user sometimes does not realizethat the batteries have run out.

In ordinary households, the installation of security cameras is becomingpopular. In this case, security cameras are generally installedoutdoors, and images from security cameras are sent to monitors disposedindoors by wireless means. This allows users to monitor the situationoutdoors through security cameras while remaining indoors.

Images captured by such security cameras are constantly recorded onrecording media, for example, video cassette tape.

Accordingly, the amount of image data recorded from the security camerasbecomes very large, and the communication cost for sending the imagedata is high. Additionally, the user has to search through a largeamount of image data sent from the security camera, which is verytime-consuming.

Thus, a method for capturing image data only when someone intrudes intoa monitored area so as to decrease the amount of image data to be storedin a storage medium has been suggested (for example, see JapaneseUnexamined Patent Application Publication No. 2000-132774). In thismethod, the communication cost can be reduced and search in playing backimage data can be facilitated.

In the method disclosed in this publication, however, when someoneintrudes into the monitored area, the camera is operated to captureimage data and the image data is stored in a storage device.Accordingly, it cannot be determined whether an intruder approaches themonitored area or goes away from the monitored area, thereby storingunnecessary images. Additionally, the standard for detecting anintrusion never changes, and events (detection of an intrusion) of nointerest to the users may sometimes be stored and reported.

SUMMARY OF THE INVENTION

Accordingly, in view of this background, it is an object of the presentinvention to flexibly detect and report events so as to inhibit powerconsumption, and also to decrease the amount of images stored in case ofthe occurrence of abnormalities.

It is another object of the present invention to more precisely detectabnormalities requested by users without increasing the powerconsumption.

It is still another object of the present invention to monitor outside avehicle by a camera disposed in the vehicle while the vehicle is beingparked, without increasing the power consumption.

In order to achieve the above object, an information processing systemof the present invention includes: a sensor unit for obtaining sensordata; a feature extracting unit for extracting a feature of an objectbased on sensor data obtained by the sensor unit; an event determiningunit for obtaining the feature extracted by the feature extracting unitso as to determine whether an event has occurred based on determinationparameters; a display-data generator for generating, when the occurrenceof the event is determined by the event determining unit, display dataincluding event data for reporting the occurrence of the event; adisplay device for giving an indication based on the display datagenerated by the display-data generator; an input reception unit forreceiving the input of a user decision based on the indication given bythe display device; and a control unit for performing control processingbased on the input of the user decision received by the input receptionunit.

The above-described display-data generator may further generateremote-controller display data consisting of the event data forreporting the occurrence of the event to a remote controller forcontrolling the display-data generator.

The aforementioned information processing system may further include aremote-controller display device for displaying the remote-controllerdisplay data generated by the display-data generator.

The control unit may include: a parameter controller for updating thedetermination parameters for determining whether the event has occurredby the event determining unit; a feature-extracting controller forcontrolling the feature extracting unit to extract the feature; and asensor controller for controlling the sensor unit to obtain the sensordata.

The sensor unit may include a plurality of sensors, and the sensorcontroller may control power sources of the plurality of sensors to beturned ON or OFF.

The display-data generator may generate data based on a predeterminedsignal as the display data when the event determining unit determinesthat an event has not occurred.

The sensor unit, the feature extracting unit, the event determiningunit, and the control unit may be formed of a first informationprocessing apparatus. The display-data generator may be formed of asecond information processing apparatus. The display device may beformed of a third information processing apparatus. The input receptionunit may be formed of a fourth information processing apparatus.

Communication between the first information processing apparatus and thesecond information processing apparatus may be performed wirelessly.

The first information processing apparatus may be driven by a battery.

The sensor unit, the feature extracting unit, the event determiningunit, and the control unit may be formed of a first informationprocessing apparatus. The display-data generator and the display devicemay be formed of a second information processing apparatus. The inputreception unit may be formed of a third information processingapparatus.

The display-data generator may generate the display data by insertingthe event data into a television broadcast signal.

The sensor unit may include at least one of a camera, a microphone, andan infrared sensor.

An information processing method of the present invention includes: asensor step of obtaining sensor data; a feature extracting step ofextracting a feature of an object based on the sensor data obtained inthe sensor step; an event determining step of obtaining the featureextracted in the feature extracting step so as to determine whether anevent has occurred based on determination parameters; a display-datagenerating step of generating, when the occurrence of the event isdetermined in the event determining step, display data including eventdata for reporting the occurrence of the event; a display step of givingan indication based on the display data generated in the display-datagenerating step; an input reception step of receiving the input of auser decision based on the indication given in the display step; and acontrol step of performing control processing based on the input of theuser decision received in the input reception step.

A program recorded in a recording medium of the present inventionincludes: a feature extracting step of extracting a feature of an objectbased on sensor data; an event determining step of obtaining the featureextracted in the feature extracting step so as to determine whether anevent has occurred based on determination parameters; a display-datagenerating step of generating, when the occurrence of the event isdetermined in the event determining step, display data including eventdata for reporting the occurrence of the event; an input reception stepof receiving the input of a user decision; and a control step ofperforming control processing based on the input of the user decisionreceived in the input reception step.

A program of the present invention allows a computer to execute: afeature extracting step of extracting a feature of an object based onsensor data; an event determining step of obtaining the featureextracted in the feature extracting step so as to determine whether anevent has occurred based on determination parameters; a display-datagenerating step of generating, when the occurrence of the event isdetermined in the event determining step, display data including eventdata for reporting the occurrence of the event; an input reception stepof receiving the input of a user decision; and a control step ofperforming control processing based on the input of the user decisionreceived in the input reception step.

An information processing apparatus of the present invention includes asensor unit for obtaining a plurality of items of sensor data by using aplurality of sensors. The sensor unit turns ON or OFF the plurality ofsensors according to a decision signal based on a user instructiontransmitted from a first information processing apparatus.

The above information processing apparatus may further include: afeature extracting unit for extracting a feature of an object based onsensor data obtained by the sensor unit; an event determining unit forobtaining the feature extracted by the feature extracting unit so as todetermine whether an event has occurred based on determinationparameters; a transmitter for transmitting, when the occurrence of theevent is determined by the event determining unit, message data forreporting the occurrence of the event to the first informationprocessing apparatus; a receiver for receiving the decision signaltransmitted from the first information processing apparatus; and acontrol unit for performing control processing based on the decisionsignal received by the receiver.

The control unit may update the determination parameters for determiningwhether the event has occurred by the event determining unit, and maycontrol the extraction processing of the feature by the featureextracting unit and may control the obtaining processing of the sensordata by the sensor unit.

Wireless communication may be performed in the transmitter and in thereceiver.

The information processing apparatus may be driven by a battery.

The plurality of sensors may include at least one of a camera, amicrophone, and an infrared sensor.

Another information processing method of the present invention is aninformation processing method for an information processing apparatuswhich includes a sensor unit for obtaining a plurality of items ofsensor data by using a plurality of sensors so as to perform monitorprocessing based on the sensor data. The information processing methodmay include the step of turning ON or OFF the plurality of sensorsaccording to a decision signal based on a user instruction transmittedfrom a first information processing apparatus.

Another information processing apparatus of the present inventionincludes a display-data generator for generating, in case of theoccurrence of an event, display data by inserting event data forreporting the occurrence of the event into data based on a predeterminedsignal. When an event has not occurred, the display-data generatorgenerates the data based on the predetermined signal as the displaydata.

The above information processing apparatus may further include a displaydevice for displaying the display data generated by the display-datagenerator.

The display-data generator may further generate remote-controllerdisplay data consisting of the event data for reporting the occurrenceof the event to a first information processing apparatus whichremote-controls the information processing apparatus.

The above information processing apparatus may further include: anevent-message receiver for receiving message data for reporting theoccurrence of the event from a second information processing apparatus;a display-data transmitter for transmitting the remote-controllerdisplay data to the first information processing apparatus; a decisionsignal receiver for receiving a decision signal based on theremote-controller display data from the first information processingapparatus; and a decision signal transmitter for transmitting thedecision signal received by the decision signal receiver to the secondinformation processing apparatus. The display-data generator may performprocessing based on the message data.

Wireless communication may be performed in the event-message receiverand in the decision signal transmitter.

The above information processing apparatus may further include: afeature extracting unit for extracting a feature of an object based onsensor data; an event determining unit for obtaining the featureextracted by the feature extracting unit so as to determine whether anevent has occurred based on determination parameters; a receiver forreceiving a decision signal based on the display data from a firstinformation processing apparatus; and a control unit for performingcontrol processing based on the decision signal received by thereceiver. The display-data generator may generate the display data basedon a determination result of the event determining unit.

The control unit may update the determination parameters for determiningwhether the event has occurred by the event determining unit, controlthe extraction processing of the feature by the feature extracting unit,and transmit a signal for controlling the obtaining processing of thesensor data by the first information processing apparatus to the firstinformation processing apparatus.

The control unit may transmit a signal for turning ON or OFF a pluralityof sensors provided for the first information processing apparatus.

Still another information processing apparatus of the present inventionincludes: a receiver for receiving remote-controller display dataconsisting of event data for reporting the occurrence of an event from afirst information processing apparatus; a display unit for displayingthe remote-controller display data received by the receiver; an inputreception unit for receiving the input of a user decision based on anindication of the display device; and a transmitter for transmitting adecision signal based on the user decision received by the inputreception unit to the first information processing apparatus.

Still another information processing method of the present inventionincludes: a reception step of receiving remote-controller display dataconsisting of event data for reporting the occurrence of an event from afirst information processing apparatus; a display control step ofcontrolling an indication of the remote-controller display data receivedin the reception step; an input reception step of receiving the input ofa user decision based on the indication controlled in the displaycontrol step; and a transmission step of transmitting a decision signalbased on the user decision received in the input reception step to thefirst information processing apparatus.

A program recorded in another recording medium of the present inventionincludes: a reception step of receiving remote-controller display dataconsisting of event data for reporting the occurrence of an event froman information processing apparatus; an input reception step ofreceiving the input of a user decision; and a transmission step oftransmitting a decision signal based on the user decision received inthe input reception step to the information processing apparatus.

A second program of the present invention allows a computer to execute:a reception step of receiving remote-controller display data consistingof event data for reporting the occurrence of an event from aninformation processing apparatus; an input reception step of receivingthe input of a user decision; and a transmission step of transmitting adecision signal based on the user decision received in the inputreception step to the information processing apparatus.

A further information processing apparatus of the present inventionincludes: a first sensor for detecting an object; a second sensor fordetecting the object; a battery for supplying power to the first sensorand the second sensor; and a power supply control unit for supplyingpower to the first sensor without supplying power to the second sensorunder normal conditions, and when an abnormality is detected by thefirst sensor, the power supply control unit supplying power to thesecond sensor.

Power consumption of the first sensor may be smaller than that of thesecond sensor.

The first sensor may be an infrared sensor, and the second sensor may bea microwave sensor.

The information processing apparatus may further include a third sensorwhose power consumption is larger than the second sensor.

The third sensor may be a video camera or a microphone.

The second sensor may be a sensor for detecting whether the object isapproaching or going away from the second sensor.

The power supply control unit does not supply power to the third sensorwhen the second sensor detects that the object is going away from thesecond sensor even if the first sensor detects the presence of theobject, and the power supply control unit may supply power to the thirdsensor when the second sensor detects that the object is approaching.

The above information processing apparatus may further include atransmitter for wirelessly transmitting an output of the third sensor.

A further information processing method of the present inventionincludes the step of controlling power supply so that power is suppliedto a first sensor without supplying power to a second sensor undernormal conditions, and when an abnormality is detected by the firstsensor, power is supplied to the second sensor.

An image-capturing device of the present invention includes: animage-capturing unit installed inside a vehicle at a position to capturean image outside the vehicle through a rear window; a direction settingunit for setting the image-capturing direction of the image-capturingunit in a monitor mode; a determining unit for determining whether thevehicle is in a first usage mode in which the vehicle is reversed or ina second usage mode in which the vehicle is not used; and a directioncontrol unit for controlling the image-capturing direction of theimage-capturing unit to be a direction in the back of the vehicle whenthe vehicle is in the first usage mode, and for controlling theimage-capturing direction to be a direction set by the direction settingunit when the vehicle is in the second usage mode.

The image-capturing device may further include: a detector for detectingan object outside the vehicle through a window of the vehicle; and apower supply control unit for prohibiting power supply to theimage-capturing unit when an object is not detected by the detector, andfor allowing power supply to the image-capturing unit when an object isdetected by the detector.

When the detector detects a moving object, the direction control unitmay control the image-capturing direction of the image-capturing unit sothat the object is traced.

The image-capturing device may further include a battery for supplyingpower to the image-capturing unit. When the remaining amount of thebattery is smaller than a reference value, the power supply control unitmay replace the battery with a battery of the vehicle to supply power.

The image-capturing device may further include: a transmitter fortransmitting an image captured by the image-capturing unit; and atransmission-destination setting unit for setting a transmissiondestination of the image captured by the image-capturing unit to a firstdisplay device disposed inside the vehicle when the vehicle is in thefirst usage mode, and for setting the transmission destination to be asecond display device disposed outside the vehicle when the vehicle isin the second usage mode.

The determining unit may determine whether the vehicle is in the firstmode or in the second mode based on the status of an engine key and thestatus of a transmission gear of the vehicle.

An image-capturing method of the present invention includes: a directionsetting step of setting an image-capturing direction of animage-capturing unit in a monitor mode; a determining step ofdetermining whether a vehicle is in a first usage mode in which thevehicle is reversed or in a second usage mode in which the vehicle isnot used; and a direction control step of controlling theimage-capturing direction of the image-capturing unit to be a directionin the back of the vehicle when the vehicle is in the first usage mode,and for controlling the image-capturing direction to be a direction setin the direction setting step when the vehicle is in the second usagemode.

A program recorded in still another recording medium of the presentinvention includes: a direction setting step of setting animage-capturing direction of an image-capturing unit in a monitor mode;a determining step of determining whether a vehicle is in a first usagemode in which the vehicle is reversed or in a second usage mode in whichthe vehicle is not used; and a direction control step of controlling theimage-capturing direction of the image-capturing unit to be a directionin the back of the vehicle when the vehicle is in the first usage mode,and for controlling the image-capturing direction to be a direction setin the direction setting step when the vehicle is in the second usagemode.

Still another program of the present invention includes: a directionsetting step of setting an image-capturing direction of animage-capturing unit in a monitor mode; a determining step ofdetermining whether a vehicle is in a first usage mode in which thevehicle is reversed or in a second usage mode in which the vehicle isnot used; and a direction control step of controlling theimage-capturing direction of the image-capturing unit to be a directionin the back of the vehicle when the vehicle is in the first usage mode,and for controlling the image-capturing direction to be a direction setin the direction setting step when the vehicle is in the second usagemode.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates the configuration of a monitor system according to anembodiment of the present invention;

FIG. 2 is a block diagram illustrating the functional configuration of amulti-sensor camera unit shown in FIG. 1;

FIG. 3 is a block diagram illustrating the functional configuration of aprocessing box shown in FIG. 1;

FIG. 4 is a block diagram illustrating the functional configuration of aremote controller shown in FIG. 1;

FIG. 5 is a block diagram illustrating the principle configuration ofthe monitor system shown in FIG. 1;

FIG. 6 is a flowchart illustrating event detection processing performedby the monitor system shown in FIG. 5;

FIG. 7 is a block diagram illustrating the configuration of themulti-sensor camera unit shown in FIG. 1;

FIG. 8 is a block diagram illustrating the configuration of theprocessing box shown in FIG. 1;

FIG. 9 is a block diagram illustrating the configuration of the remotecontroller shown in FIG. 1;

FIGS. 10 and 11 are a flowchart illustrating processing performed by themulti-sensor camera unit shown in FIG. 7;

FIGS. 12 and 13 are a flowchart illustrating processing performed by theprocessing box shown in FIG. 8;

FIG. 14 is a flowchart illustrating processing performed by the remotecontroller shown in FIG. 9;

FIGS. 15 and 16 illustrate display examples in the processing of stepS104 of FIG. 12;

FIGS. 17 and 18 illustrate display examples in the processing of stepS153 of FIG. 14;

FIG. 19 is a bock diagram illustrating the configuration of a personalcomputer;

FIG. 20 illustrates the overall configuration of a security systemaccording to another embodiment of the present invention;

FIG. 21 is a block diagram illustrating the configuration of a securitycamera unit shown in FIG. 20;

FIG. 22 is a block diagram illustrating the functional configuration ofthe security camera unit shown in FIG. 20;

FIG. 23 is a flowchart illustrating object detection processingperformed by the security camera unit shown in FIG. 20;

FIGS. 24 and 25 illustrate outputs of an infrared sensor and a microwavesensor shown in FIG. 21;

FIG. 26 illustrates an image-capturing device in a back monitor modeaccording to still another embodiment of the present invention;

FIG. 27 illustrates an example of the installment of the image-capturingdevice shown in FIG. 26;

FIG. 28 is a block diagram illustrating the electrical configuration ofthe image-capturing device shown in FIG. 26;

FIG. 29 is a block diagram illustrating the electrical configuration ofa reception display device disposed inside the vehicle shown in FIG. 26;

FIG. 30 is a block diagram illustrating the electrical configuration ofa reception display device disposed outside the vehicle shown in FIG.26;

FIG. 31 is a flowchart illustrating monitoring-directionpre-registration processing;

FIG. 32 illustrates the image-capturing device shown in FIG. 26 in amonitor mode;

FIG. 33 is a flowchart illustrating sensor processing;

FIG. 34 is a flowchart illustrating back monitor processing; and

FIG. 35 is a flowchart illustrating monitor mode processing.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present invention is described in detail below with reference to thedrawings through illustration of preferred embodiments.

FIG. 1 illustrates the configuration of a monitor system 10 according toan embodiment of the present invention. In the monitor system 10, amulti-sensor camera unit 1 is disposed in a monitored area at the leftside of FIG. 1, and a processing box 2, a display device 3, and a remotecontroller 4 for remote-controlling the processing box 2 are provided ata report/display portion at the right side of FIG. 1. Wirelesscommunication is performed between the multi-sensor camera unit 1 andthe processing box 2.

A plurality of sensors, driven by batteries, are provided in themulti-sensor camera unit 1. The sensors are installed in an area formonitoring events. When an event occurs, the processing box 2 constructsan image to be displayed, and sound to be output, informs the displaydevice 3 and the remote controller 4 that an event has occurred, andcontrols the display device 3 and the remote controller 4 to display theimage and output sound. The display device 3 may be, for example, ageneral television receiver. In this case, when there is no event(normal), the display device 3 displays general audio-visual signals(video images based on broadcast signals), and when an event occurs, thedisplay device 3 displays a picture-in-picture image in which an eventimage is inserted into part of the general audio-visual signal. Thedisplay device 3 is not restricted to a television receiver, and may bea dedicated monitor. The image to be displayed does not have to be apicture-in-picture image, and the image may be displayed on the entirescreen.

A user makes a decision for the displayed event, and inputs aninstruction through the remote controller 4 based on the result ofhis/her decision. For example, if the user wishes that the eventcontinue to be reported from now on, he/she inputs a correspondinginstruction by operating an OK button (OK button 291 shown in FIG. 9,which is described below). Events detected by the processing box 2change over time based on the instructions input by the user, and onlythe events desired by the user are detected and reported every time theuser uses the monitor system 10.

The sensors and a signal processor (not shown) loaded in themulti-sensor camera unit 1 are operated only when necessary, therebypreventing wasteful power consumption.

FIG. 2 is a block diagram illustrating the functional configuration ofthe multi-sensor camera unit 1 shown in FIG. 1.

The multi-sensor camera unit 1 is provided with a camera 21, amicrophone 22, an infrared sensor 23, other sensors 24, a signalprocessor 25, a transmitter 26, a receiver 27, and batteries 28.

The signal processor 25 detects an event based on data obtained from aplurality of sensors, i.e., the infrared sensor 23 and the other sensors24, and in case of an event, the signal processor 25 transmits datarequired for showing the event to the processing box 2 via thetransmitter 26 (informs the processing box 2 of the event). If one ofthe other sensors 24 is, for example, a photo sensor, it senses that alight is turned on indoors. If one of the other sensors 24 is amicrowave sensor, it detects the moving distance and the motion of amoving body. The batteries 28 supply required power to the individualelements of the multi-sensor camera unit 1.

The receiver 27 receives a user decision signal, a sensor controlsignal, or a signal processing control signal sent from the processingbox 2 in accordance with an event, and supplies the signals to thesignal processor 25. The signal processor 25 performs processing basedon the received signals. The multi-sensor camera unit 1 is able toobtain a user decision so as to operate the sensors, the signalprocessor 25, the transmitter 26, and the receiver 27 required fordetecting and displaying an event, thereby reducing the powerconsumption of the multi-sensor camera unit 1.

Since the multi-sensor camera unit 1 is small and battery-driven, andalso has a wireless communication function, it can be installed invarious places indoors without the need for a large space or for wiringfor supplying power. Since it is provided with a plurality of sensors,the multi-sensor camera unit 1 is also able to detect various eventsindoors.

FIG. 3 is a block diagram illustrating the functional configuration ofthe processing box 2 shown in FIG. 1.

A receiver 51 receives signals sent from the transmitter 26 of themulti-sensor camera unit 1 by wireless means (radio waves), and suppliesthem to a signal processor 52. The signal processor 52 receives generalaudio-visual signals (television broadcast signals), and also performsprocessing or constructs an image to be displayed based on the signalsupplied from the receiver 51. In case of an event, the signal processor52 constructs data to be displayed on the display device 3, and outputsit to the display device 3, and also constructs data to be displayed onthe remote controller 4, and sends it to the remote controller 4 via atransmitter 53 by wireless means (infrared or radio waves).

A receiver 54 receives a signal indicating a user decision (userdecision signal) transmitted from the remote controller 4 by wirelessmeans (infrared or radio waves), and supplies the signal to the signalprocessor 52. The signal processor 52 performs processing based on theuser decision signal, and also sends required data to the multi-sensorcamera unit 1 via a transmitter 55 by wireless means (radio waves).

FIG. 4 is a block diagram illustrating the functional configuration ofthe remote controller 4 shown in FIG. 1.

A receiver 81 receives a signal (for example, remote-controller displaydata) sent from the transmitter 53 of the processing box 2 by wirelessmeans (infrared or radio waves), and supplies the signal to a signalprocessor 82. The signal processor 82 performs processing, such asdecoding or coding, on the received signal, based on the signal.

A remote-controller display device 83 displays an image (includingcharacters and symbols) based on the signal sent from the signalprocessor 82. A user input interface (IF) 84 receives the input of auser decision for an event, and supplies a corresponding signal to thesignal processor 82.

In case of an event, the receiver 81 receives remote-controller displaydata sent from the transmitter 53 of the processing box 2, and thesignal processor 82 controls the remote-controller display device 83 todisplay an image indicating the display data. When a user decision isinput into the user input IF 84, a user decision signal is supplied tothe signal processor 82. The signal processor 82 sends the user decisionsignal to the processing box 2 via a transmitter 85 by wireless means(infrared or radio waves).

Two buttons (for example, the OK button 291 and an NO button 292 shownin FIG. 9, which are described below) are disposed on the remotecontroller 4 so that the user is able to input a user decision for anevent, for example, “continue to report” or “no need to continuereporting”. Based on the input of the user decision, the multi-sensorcamera unit 1 and the processing box 2 change the processing performed.

The principle configuration obtained by connecting the main elements ofthe monitor system 10 shown in FIG. 1 formed of the multi-sensor cameraunit 1, the processing box 2, and the remote controller 4 shown in FIGS.2, 3, and 4, respectively, is shown in FIG. 5. In FIG. 5, wirelesscommunication channels are also indicated as wired communicationchannels.

A sensor unit 101 formed of plurality of sensors 101 a through 101 ncorresponds to the multi-sensor camera unit 1 shown in FIG. 2. A featureextracting unit 102, an event determining unit 103, a display dataconstructor 104, and a control unit 108 including a parameter controller121, a signal processing controller 122, and a power controller 123correspond to the processing box 2 shown in FIG. 3. A remote-controllerdisplay device 83 and a user input IF 84 correspond to the remotecontroller 4 shown in FIG. 4. The correlations between the main elementsshown in FIG. 1 and the elements shown in FIG. 5 are not restricted tothe above-described arrangements; for example, the feature extractingunit 102, the event determining unit 103, and part of or the wholecontrol unit 108 may be provided in the multi-sensor camera unit 1 (sucha configuration is shown in FIG. 7).

The processing performed by the monitor system 10 shown in FIG. 5 isdescribed below with reference to the flowchart of FIG. 6. Thisprocessing starts when the user gives an instruction to start monitoringa monitored area.

In step S1, the feature extracting unit 102 obtains sensor data fromeach of the sensors 101 a through 101 n forming the sensor unit 101disposed in the monitored area.

In step S2, the feature extracting unit 102 calculates a feature fromeach type of sensor data obtained in step S1. More specifically, thefeature extracting unit 102 determines, for example, an inter-framedifference of an image signal, so as to calculate the center of gravityof the inter-frame difference indicating the position of a movingobject. Alternatively, the feature extracting unit 102 calculates theamount by which the data of an infrared sensor changes.

In step S3, the event determining unit 103 obtains the featurescalculated by the feature extracting unit 102.

In step S4, the event determining unit 103 determines whether an eventhas occurred based on each feature obtained in step S3. In this case,the event determining unit 103 determines whether an event has occurredbased on event-determining parameters possessed by the event determiningunit 103. The parameters are updated under the control of the parametercontroller 121. If it is determined in step S4 that an event hasoccurred, the process proceeds to step S5. In step S5, the eventdetermining unit 103 supplies an event occurrence signal indicating thatan event has occurred and data required for reporting and displaying theevent to the display data constructor 104.

In step S6, the display data constructor 104 obtains the eventoccurrence signal and data required for reporting and displaying theevent from the event determining unit 103, and constructs display datain which event display data is inserted into part of a generalaudio-visual signal (television broadcast signal). The display dataconstructor 104 also constructs display data for the remote controller 4(hereinafter referred to as “remote-controller display data”) formed ofthe event display data without the general audio-visual signal.

In step S7, the display data constructor 104 outputs the display dataconstructed in step S6 to the display device 3, and controls the displaydevice 3 to display the data. The display data constructor 104 alsooutputs the remote-controller display data to the remote-controllerdisplay device 83, and controls the remote-controller display device 83to display the remote-controller display data.

Since the display data is formed by inserting event display data intopart of a general audio-visual signal, a picture-in-picture image, suchas that shown in FIG. 15, which is described below, is displayed on thedisplay device 3. Since the remote-controller display data is formed ofonly event display data, an image indicating an event only (for example,an image of the place where monitoring is conducted) is displayed on theremote-controller display device 83.

By viewing the event (image and sound) reported and displayed on theremote-controller display device 83, the user inputs a decision for theevent, for example, whether or not the user wishes that the eventcontinue to be reported. In this case, an image for instructing the userto input a decision may be displayed on the remote-controller displaydevice 83. When a decision is input from the user, in step S8, the userinput IF 84 obtains the input of the user decision for the event. Theuser input IF 84 then supplies a user decision signal indicating theuser decision to the control unit 108 formed of the parameter controller121, the signal processing controller 122, and the power controller 123.

In step S9, the parameter controller 121 updates event determiningparameters possessed by the event determining unit 103 based on the userdecision signal obtained in step S8 so that events can be detected inresponse to the user instruction. For example, if the parameters are setsuch that an event occurrence signal is generated when a brightnesshaving a level of a reference value or higher is detected, the referencevalue can be changed to be higher or lower.

In step S10, the signal processing controller 122 controls the featureextracting unit 102 based on the user decision signal obtained in stepS8. More specifically, the signal processing controller 122 controls thefeature extracting unit 102 to stop detecting unnecessary features or toswitch from lighter processing (for example, detecting smaller number offeatures) to heavier processing (for example, detecting larger number offeatures) so as to perform more precise detection. In FIG. 15, forexample, for a portion 322, which should be detected as an event, and aportion 323, which should not be detected as an event, the signalprocessing controller 122 controls the feature extracting unit 102 so asnot to detect the portion 323 as a feature even though the image of aperson is captured and contained in the portion 323.

In step S11, the power controller 123 controls the power source of thesensor unit 101 to be turned ON or OFF based on the user decision signalobtained in step S8. More specifically, among the sensors 101 a through101 n, the power controller 123 controls the power sources of thesensors unnecessary for the feature extracting unit 102 and for theevent determining unit 103 to be turned OFF, thereby preventingunnecessary consumption of the batteries.

After step S11, the process returns to step S1, and processing similarto the above-described processing is repeated.

If the event determining unit 103 determines in step S4 that an eventhas not occurred, the process proceeds to step S12. In step S12, thedisplay data constructor 104 outputs a general audio-visual signal(television broadcast signal) to the display device 3 as the displaydata, and the display device 3 displays the general audio-visual signal.Then, the process returns to step S1, and step S1 and the subsequentsteps are then repeated.

According to the processing shown in FIG. 6, by inputting a simpledecision, the user is able to reliably detect only desired events, andthe sensors and the processors are operated only when necessary, therebyreducing the power consumption of the multi-sensor camera unit 1.

A specific example of the monitor system (home security system) 10 isdescribed below with reference to FIGS. 7 through 18.

In this example, a camera 201 and an infrared sensor 202 (FIG. 7) areused as sensors, and the user inputs an instruction for a displayedevent by operating the OK button 291 indicating “OK (needs to bedetected from now on)” or the NO button 292 indicating “NO (no need tobe detected)”. In this case, the configuration of the multi-sensorcamera unit 1 is shown in FIG. 7, the configuration of the processingbox 2 is shown in FIG. 8, and the configuration of the remote controller4 is shown in FIG. 9. The processing by the multi-sensor camera unit 1is shown in FIGS. 10 and 11, the processing by the processing box 2 isshown in FIGS. 12 and 13, and the processing by the remote controller 14is shown in FIG. 14. Images displayed on the display device 3 are shownin FIGS. 15 and 16. Images displayed on the remote-controller displaydevice 83 are shown in FIGS. 17 and 18. The same elements as those shownin FIG. 5 are indicated by like reference numerals, and an explanationthereof is thus omitted.

In FIG. 7, sensor data obtained by the camera 201 and the infraredsensor 202 is supplied to the feature extracting unit 102. Under thecontrol of the control unit 108, for example, the infrared sensor 202 isalways turned ON and the camera 201 is normally turned OFF to reducepower consumption. The feature extracting unit 102 extracts featuresfrom the sensor data obtained by the infrared sensor 202, and suppliesthem to the event determining unit 103.

Message data used for reporting the occurrence of an event is suppliedfrom the feature extracting unit 102 to a coder 203. The coder 203 codesthe message data and supplies it to a transmitter 204. In case of theoccurrence of an event, the transmitter 204 sends the coded message datato the processing box 2. A user decision signal sent from the processingbox 2 is received by a receiver 205, decoded by a decoder 206, and thensupplied to the control unit 108. The control unit 108 performsprocessing based on the user decision signal, as described above.

More specifically, as described with reference to FIG. 5, the controlunit 108 controls the event determining unit 103 to update the eventdetermination parameters, or controls the feature extracting unit 102 tostart or stop extracting the features of an image, or the camera 201 orthe infrared sensor 202 to be turned ON or OFF.

The coder 203, the transmitter 204, the receiver 205, and the decoder206 are also controlled to perform processing only when an event occurs.Accordingly, the multi-sensor camera unit 1 can be operated withoutwastefully consuming power.

When an event message (message data) is sent from the transmitter 204 ofthe multi-sensor camera unit 1, it is received by a receiver 241 of theprocessing box 2 shown in FIG. 8. The message data is decoded by adecoder 242, and is then supplied to the display data constructor 104.As stated above, the display data constructor 104 constructs displaydata by superposing an event signal on a general audio-visual signal,and outputs the display data to the display device 3. The display dataconstructor 104 also controls a coder 243 to code the event data itselfas the remote-controller display data, and controls a transmitter 244 tosend the remote-controller display data. When an event message (messagedata) is not sent from the multi-sensor camera unit 1, i.e., when thereis no event, the display data constructor 104 outputs a generalaudio-visual signal to the display device 3 as the display data, andcontrols the display device 3 to display it.

After sending the remote-controller display data, a user decision signalis returned form the remote controller 4. Then, a receiver 245 of theprocessing box 2 receives the user decision signal and supplies it to adecoder 246. The decoder 246 decodes the user decision signal, andsupplies it to the display data constructor 104 and a coder 247. Thedisplay data constructor 104 performs, for example, processing forstopping generating a picture-in-picture image, based on the userdecision signal. The coder 247 codes the supplied user decision signal,and supplies it to a transmitter 248. The user decision signal istransmitted from the transmitter 248 and is received by the receiver 205of the multi-sensor camera unit 1.

When the remote-controller display data is sent from the transmitter 244of the processing box 2, a receiver 281 of the remote controller 4 shownin FIG. 9 receives the remote-controller display data, and supplies itto a decoder 282. The decoder 282 decodes the remote-controller displaydata, and controls the remote-controller display device 83 to display animage based on the decoded remote-controller display data.

The user then operates the OK button 291 or the NO button 292 to input auser decision indicating whether the corresponding event is to bedetected.

The user input IF 84 detects that the OK button 291 or the NO button 292has been operated, and supplies a detection result to a coder 283 as auser decision signal. The coder 283 codes the user decision signal andsends it to a transmitter 284. The transmitter 284 then sends the userdecision signal to the receiver 245 of the processing box 2.

The processing by the multi-sensor camera unit 1 shown in FIG. 7, theprocessing by the processing box 2 shown in FIG. 8, and the processingby the remote controller 4 shown in FIG. 9 are described below withreference to the flowcharts of FIGS. 10 through 14.

A description is first given of the processing by the multi-sensorcamera unit 1 shown in FIG. 7 with reference to the flowcharts of FIGS.10 and 11. This processing starts when a user gives an instruction tostart monitoring a monitored area.

In step S51, the camera 201 and the infrared sensor 202, which areconsidered to be necessary sensors, obtain image data and detectiondata, respectively, as sensor data, and supply them to the featureextracting unit 102. Although in this example both the camera 201 andthe infrared sensor 202 are operated, only the infrared sensor 102 oronly the camera 201 may be operated.

In step S52, the feature extracting unit 102 calculates the featuresrequired. More specifically, the feature extracting unit 102 calculatesthe features required, for example, an inter-frame difference of imagedata output from the camera 201, the center of gravity of theinter-frame difference of the image data, and an amount by which thesensor data output from the infrared sensor 202 changes. The featuresrequired change from time to time under the control of the control unit108. The feature extracting unit 102 supplies the calculated features tothe event determining unit 103, and also supplies them to the coder 203as message data.

In step S53, the event determining unit 103 obtains the featurescalculated and supplied in step S52.

In step S54, the event determining unit 103 determines whether an eventhas occurred based on the obtained features. In this case, the eventdetermining unit 103 determines the occurrence of an event based on theevent determining parameters possessed in the event determining unit103. The parameters are updated under the control of the control unit108. If it is determined in step S54 that an event has occurred, theprocess proceeds to step S55. In step S55, the event determining unit103 sends an event occurrence signal to the coder 203, the transmitter204, the receiver 205, and the decoder 206. Upon receiving the eventoccurrence signal, the coder 203, the transmitter 204, the receiver 205,and the decoder 206 become operable. Accordingly, when an eventoccurrence signal is not supplied, the above elements are not operable(OFF), thereby inhibiting wasteful power consumption.

In step S56, the coder 203 receives and codes the message data sent fromthe feature extracting unit 102 in step S52, and supplies the codedmessage data to the transmitter 204.

In step S57, the transmitter 204 sends the message data supplied andcoded by the coder 203 to the processing box 2.

The processing box 2 receives the message data (in step S103 of FIG. 12,which is described below), and returns a user decision signal inresponse to this message data (in step S112 of FIG. 13, which isdescribed below).

In step S58, the receiver 205 determines whether a user decision signalhas been returned. If the user decision signal has been returned, theprocess proceeds to step S59. In step S59, the receiver 205 receives theuser decision signal and supplies it to the decoder 206.

In step S60, the decoder 206 decodes the user decision signal andsupplies it to the control unit 108.

In step S61, the control unit 108 controls the parameter updatingprocessing of the event determining unit 103, the signal processing ofthe feature extracting unit 102, and the power supply to the sensors.Details have been described above with reference to FIGS. 5 and 6, andan explanation thereof is thus omitted.

If it is determined in step S54 that an event has not occurred, or if itis determined in step S58 that a user decision signal has not beenreturned, or after step S61, the process returns to step S51, and stepS51 and the subsequent steps are then repeated.

The processing performed by the processing box 2 shown in FIG. 8corresponding to the processing by the multi-sensor camera unit 1 shownin FIGS. 10 and 11 is now described with reference to FIGS. 12 and 13.This processing starts when a user gives an instruction to startdisplaying an image corresponding to a general audio-visual signal(broadcast program signal) on the display device 3 or to startmonitoring a monitored area.

In step S101, the receiver 241 determines whether an event message hasbeen received. More specifically, the receiver 241 determines whethermessage data has been sent from the transmitter 204 of the multi-sensorcamera unit 1. If it is determined in step S101 that an event message(message data) has not been sent, the process proceeds to step S102. Instep S102, the display data constructor 104 outputs a generalaudio-visual signal to the display device 3 as the display data, andcontrols the display device 3 to display it. Thereafter, the processreturns to step S101, and step S101 and the subsequent steps are thenrepeated.

If it is found in step S101 that an event message has been sent, theprocess proceeds to step S103. In step S103, the receiver 241 receivesthe message data and outputs it to the decoder 242. The decoder 242decodes the message data, and supplies the decoded message data to thedisplay data constructor 104.

In step S104, the display data constructor 104 receives the message dataand constructs display data. More specifically, the display dataconstructor 104 constructs display data (picture-in-picture image) bysuperposing an event image on part of a general audio-visual signal.Then, the display data constructor 104 outputs the display data to thedisplay device 3, and controls the display device 3 to display it.

In this case, an image shown in FIG. 15 or FIG. 16 is displayed on thedisplay device 3. In the example shown in FIG. 15, an event display area321 is displayed as part of a general audio-visual screen 320 on thedisplay device 3. Between the left-side portion 322 and the right-sideportion 323 of the event display area 321, an image of a person capturedby the camera 201 is contained in the right-side portion 323. Inresponse to this event display, the user operates the OK button 291indicating “need to be detected” or the NO button 292 indicating “noneed to be detected” of the remote controller 4 so as to input a userdecision. The resulting user decision signal is returned from the remotecontroller 4 (step S156 of FIG. 14, which is described below).

If the user wishes that the left-side portion 322 be detected as anevent rather than the right-side portion 323, the user operates the NObutton 292 for the event display shown in FIG. 15. Then, a user decisionsignal corresponding to the operation of the NO button 292 is returnedfrom the remote controller 4. The processing box 2 receives the userdecision signal and sends it to the multi-sensor camera unit 1 (stepS112 of FIG. 13, which is described below). As stated above, uponreceiving this user decision signal, the control unit 108 of themulti-sensor camera unit 1 controls the feature extracting unit 102 toexclude the right-side area of the viewing angle of the camera 201 fromthe feature extracting area (step S61 of FIG. 11). As a result, even ifan image of a person is captured in the right-side area of the viewingangle, an event occurrence signal is not generated, but when an image ofa person is captured in the left-side area of the viewing angle, anevent message is sent. Then, an image, such as that shown in FIG. 16, isdisplayed.

When the user operates the NO button 292, only a general audio-visualimage is displayed on the display device 3 without displaying the eventdisplay area 321. Accordingly, the user is able to view only the desiredareas on the screen.

Returning to FIG. 12, in step S105, the display data constructor 104constructs remote-controller display data and outputs it to the coder243. The remote-controller display data is formed of only message datawithout containing a general audio-visual signal. Upon receiving theremote-controller display data, the remote controller 4 displays acorresponding image, such as that shown in FIG. 17 or 18 (step S153 ofFIG. 14, which is described below).

In step S106, the coder 243 codes the remote-controller display datasupplied in step S105, and supplies it to the transmitter 244.

In step S107, the transmitter 244 sends the remote-controller displaydata coded and supplied in step S106 to the remote controller 4. Uponreceiving the coded remote-controller display data (step S151 of FIG.14, which is described below), the remote controller 4 displays it (stepS153, which is described below), and then, returns a user decisionsignal based on the user decision (step S156 of FIG. 14, which isdescribed below).

Then, in step S108, the receiver 245 determines whether a user decisionsignal has been returned. If so, in step S109, the receiver 245 receivesthe user decision signal and supplies it to the decoder 246.

In step S110, the decoder 246 decodes the user decision signal suppliedin step S109, and supplies it to the display data constructor 104 andalso to the coder 247.

In step S111, the coder 247 codes the user decision signal supplied instep S110, and supplies it to the transmitter 248.

In step S112, the transmitter 248 sends the user decision signal codedand supplied in step S111 to the multi-sensor camera unit 1. Themulti-sensor camera unit 1 performs processing based on the userdecision signal, as discussed above (step S61 of FIG. 11).

In step S113, the display data constructor 104 obtains the user decisionsignal decoded in step S110, and performs processing based on the userdecision signal.

In this case, as described above, if the user decision signal containsan instruction to stop constructing display data (when the NO button 292shown in FIG. 17 is operated), the display data constructor 104 stopsconstructing display data including event images. In this case, theevent display area 321 is not displayed on the display device 3.

If the user decision signal contains an instruction to continueconstructing image data (when the OK button 291 shown in FIG. 17 isoperated), the display data constructor 104 continues constructingdisplay data including event images. In this case, the event displayarea 321, such as that shown in FIG. 16, is displayed on the displaydevice 3.

After step S102, or if it is determined in step S108 that a userdecision signal has not been returned, or after step S113, the processreturns to step S101, and step S101 and the subsequent steps are thenrepeated.

A description is now given, with reference to FIG. 14, of the processingperformed by the remote controller 4 shown in FIG. 9 corresponding tothe processing by the processing box 2 shown in FIGS. 12 and 13. Thisprocessing starts when the transmitter 244 of the processing box 2performs the processing in step S107 of FIG. 12.

In step S151, the receiver 281 receives the remote-controller displaydata sent from the transmitter 244 of the processing box 2 in step S107of FIG. 12, and supplies it to the decoder 282.

In step S152, the decoder 282 decodes the remote-controller display datareceived in step S151, and supplies the decoded data to theremote-controller display device 83.

In step S153, the remote-controller display device 83 displays an imagebased on the received remote-controller display data.

In this case, an image, such as that shown in FIG. 17 or 18, isdisplayed on the remote-controller display device 83. This image issimilar to that of the event display area 321 shown in FIGS. 15 and 16.That is, the remote-controller display data contains data onlycorresponding to the event display area 321 of the display data outputto the display device 3.

In the example shown in FIG. 17, an event based on the remote-controllerdisplay data is displayed on the remote-controller display device 83.Between a left-side portion 293 and a right-side portion 294 of theimage displayed on the remote-controller display device 83, if the userwishes that the left-side portion 293 be detected as an event, the useroperates the NO button 292 for the right-side portion 294 containing animage of a person, as shown in FIG. 17. In contrast, the user operatesthe OK button 291 for the left-side portion 293 containing an image of aperson, as shown in FIG. 18.

In step S154, the user input IF 84 determines whether a user decision isinput, i.e., whether the OK button 291 or the NO button 292 is operated.If it is determined that a user decision is input, the process proceedsto step S155. In step S155, the user input IF 84 supplies a userdecision signal corresponding to the user decision to the coder 283, andthe coder 283 codes the user decision signal. Then, the coder 283supplies the coded user decision signal to the transmitter 284.

In step S156, the transmitter 284 sends the user decision signal codedin step S155 to the processing box 2. The multi-sensor camera unit 1 andthe processing box 2 receive the user decision signal (in step S59 ofFIG. 11 and in step S109 of FIG. 12, respectively), and performprocessing based on the user decision signal, as described above. Forexample, if the user decision signal indicating that the user hasoperated the NO button 292 for the display shown in FIG. 17, themulti-sensor camera unit 1 excludes the right-side area of the viewingangle (range) to be captured by the camera 201 from the featureextracting area. As a result, an image containing a person at theright-side portion 294, such as that shown in FIG. 17, is not sent, andan image containing a person at the left-side portion 293, such as thatshown in FIG. 18, is displayed as an event image.

If it is determined in step S154 that a user decision is not input, orafter step S156, the process returns to step S151, and step S151 and thesubsequent steps are then repeated.

The processing of the multi-sensor camera unit 1 may be changed, forexample, as follows. The camera 201 and the infrared sensor 202 arecontrolled to be always turned ON during a predetermined period (whenthe number of user decisions input is lower than a predetermined number)after the start of the use of the monitor system 10. Then, after thelapse of the predetermined period (when the number of user decisionsinput reaches the predetermined number), the camera 201 is turned OFF,and, when the infrared sensor 202 is operated, the camera 201 is turnedON.

The event determining unit 103 may change the determination parameters,for example, as follows. The event determining unit 103 outputs an eventoccurrence signal for any image containing a human (intruder) within thepredetermined period (when the number of user decisions input is lowerthan a predetermined number) after the start of the use of the monitorsystem 10. After the lapse of the predetermined period (when the numberof user decisions input reaches the predetermined number), the eventdetermining unit 103 generates an event occurrence signal only when animage of a human (intruder) is contained in a position designated by theuser by operating the OK button 291.

According to the above-described processing, the sensors to be operatedcan be switched based on user decisions, and the processing by themulti-sensor camera unit 1 can be changed by repeatedly inputting userdecisions. Accordingly, only events desired by the user can be detectedand reported, and also, sensors and signal processors required only areoperated so as to flexibly detect and report events, thereby inhibitingwasteful power consumption. Since the multi-sensor camera unit 1 isdriven by the batteries 28, it is preferable that at least one of thefeature extracting unit 102, the event determining unit 103, and thecontrol unit 108 be housed in the processing box 2 in order to reducethe power consumption.

The size of the multi-sensor camera unit 1 can also be reduced, andaccordingly, it can be easily installed in various places.

The above-described configuration is only an example to implement themonitor system 10. Other system configurations can be considered, andexamples thereof are described below.

Sensors used in the monitor system 10 are not restricted to cameras,microphones, and infrared sensors, and may be other types of sensors.Events to be detected are not limited to the above-described examples.

Communication between the multi-sensor camera unit 1 and the processingbox 2 does not have to be performed by wireless means, and wired meansmay be used.

A plurality of multi-sensor camera units 1 may be provided, and aplurality of display devices 3 may be disposed. Although the processingbox 2 and the display device 3 are separately provided in theabove-described embodiment, they may be integrated into one housing.

The remote controller 4 does not have to be provided with theremote-controller display device 83, and only the display device 3 maybe disposed. Instead of displaying events on the remote-controllerdisplay device 83, a display device and an input IF for inputting userdecisions may be provided in the processing box 2.

The above-described series of processings may be executed by hardware orsoftware. If software is used, the processings are executed by acomputer in which a corresponding software program is integrated intodedicated hardware. Alternatively, the corresponding software program isinstalled from a recording medium into a computer, for example, ageneral-purpose computer that is able to execute various functions byinstalling various programs. In this case, the above-describedprocessings are performed by a computer, such as a personal computer 500shown in FIG. 19.

In FIG. 19, a central processing unit (CPU) 501 executes various typesof processings according to a program stored in a read only memory (ROM)502 or a program loaded into a random access memory (RAM) 503 from astorage unit 508. In the RAM 503, data required for performing varioustypes of processings by the CPU 501 are also stored.

The CPU 501, the ROM 502, and the RAM 503 are connected to each othervia an internal bus 504. An input/output interface 505 is also connectedto the internal bus 504.

The input/output interface 505 is connected to an input unit 506including a keyboard, a mouse, etc., an output unit 507 including adisplay device, for example, a cathode ray tube (CRT) display or aliquid crystal display (LCD), and a speaker, the storage unit 508including a hard disk, and a communication unit 509 including a modem, aterminal adapter, etc. The communication unit 509 performs communicationvia various networks including telephone lines and cable television.

A drive 510 is connected to the input/output interface 505 if necessary,and a removable medium 521 formed of a magnetic disk, an optical disc, amagneto-optical disk, or a semiconductor memory is loaded in the drive510. A computer program read from the removable medium 521 is installedinto the storage unit 508 according to the necessity.

If software is used, the processings are executed by a computer in whicha corresponding software program is integrated into dedicated hardware.Alternatively, the corresponding software program is installed from arecording medium into a computer, for example, a general-purposecomputer that is able to execute various functions by installing variousprograms.

Such a recording medium includes not only a package medium formed of theremovable medium 521 recording the program therein, as shown in FIG. 19,which is distributed to the user separately from the computer, but alsothe ROM 502 or a hard disk contained in the storage unit 508 recordingthe program therein, which is provided to the user by being integratedinto the computer.

Steps of the computer program may be performed in chronological orderdescribed in this embodiment, and also may be performed concurrently orindividually.

In this embodiment, the system means the overall apparatus consisting ofa plurality of devices.

As described above, according to this embodiment, events can bereported, and more particularly, events can be flexibly detected andreported based on the input of user decisions while inhibiting wastefulpower consumption.

When no event has occurred, a predetermined signal may be sent to theuser, and in case of the occurrence of an event, the correspondinginformation may be given to the user.

Another embodiment of the present invention is described below.

FIG. 20 illustrates the overall configuration of a security system 401according to another embodiment of the present invention.

The security system 401 is formed of a security camera unit 411 and areceiver 412. The security camera unit 411 is installed outdoors (forexample, in the porch or in the garden), and the receiver 412 isinstalled indoors (for example, at the entrance or in the living room).The security camera unit 411 and the receiver 412 can communicate witheach other by wireless means, as in the previous embodiment. Thesecurity camera unit 411 sends an image signal and an audio signalobtained by a built-in video camera 422 and a built-in microphone 423,which are described below with reference to FIG. 21, to the receiver412.

The receiver 412 receives the image signal and the audio signal from thesecurity camera unit 411, outputs them to a display device, such as anLCD device, and a speaker, respectively, (neither of them is shown),provided for the receiver 412, and controls the display device and thespeaker to display the image signal and to output the audio signal,respectively. The user is then able to check whether there is anintruder outdoors while remaining indoors.

FIG. 21 is a block diagram illustrating the configuration of thesecurity camera unit 411. The security camera unit 411 includes acontrol unit 421, the video camera (may be a digital camera) 422, themicrophone 423, an infrared sensor 424, a microwave sensor 425, acommunication unit 426, and a power supply unit 427. Batteries 431supply power to the individual elements of the security camera unit 411.

The control unit 421 formed of, for example, a microcomputer, controlsthe operations of the video camera 422, the microphone 423, the infraredsensor 424, the microwave sensor 425, the communication unit 426, andthe power supply unit 427. The functional configuration of the controlunit 421 is described below with reference to FIG. 22.

The video camera 422 captures images in the photographic area bymonitoring the situation outdoors (for example, the situation of a porchor a garden). If an object, such as an intruder, enters the photographicarea, the image of such an intruder is captured. The microphone 423collects sound issued from and near an intruder, for example, anintruder's voice or the sound of intruder's action, and the sound of anobject broken by the intruder, and converts the sound into an electricsignal so as to supply it to the control unit 421.

The infrared sensor 424 receives light in an infrared area emitted froman object (not shown), and converts the light into an electric signal soas to supply it to the control unit 421. The microwave sensor 425generates microwaves and detects reflection waves generated when themicrowaves are reflected by an object. The microwave signal 425 thengenerates a detection signal indicating whether the reflection waveslead or lag with respect to a reference phase, and supplies thedetection signal to the control unit 421. The phase lead and phase lagare due to the Doppler effect, and correspond to the situations where anobject approaches the microwave sensor 425 and an object goes away fromthe microwave sensor 425.

The communication unit 426 obtains an image signal supplied from thevideo camera 422 or an audio signal supplied from the microphone 423based on a communication control signal supplied from the control unit421, and sends the corresponding signal to a communication device (notshown) of the receiver 412. The power supply unit 427 supplies powerfrom the batteries 431 to the video camera 422, the microphone 423, theinfrared sensor 424, the microwave sensor 425, and the communicationunit 426 under the control of the control unit 421. The batteries 431may be primary cells or secondary cells.

The levels of power consumption of the elements of the security cameraunit 411 decrease in the following order of: the video camera 422, themicrophone 423, the microwave sensor 425, and the infrared sensor 424 inorder of decreasing power consumption.

FIG. 22 is a block diagram illustrating the functional configuration ofthe security camera unit 411 shown in FIG. 20. The same elements asthose shown in FIG. 21 are indicated by like reference numerals, and anexplanation thereof is thus omitted. The control unit 421 includes anabnormality detector 451 and a power controller 452.

The abnormality detector 451 determines whether an abnormality hasoccurred based on a detection signal supplied from the infrared sensor424, and also based on a detection signal supplied from the microwavesensor 425. Based on a determination result indicating whether anabnormality has been detected by the infrared sensor 424 or themicrowave sensor 425, the abnormality detector 451 generates amicrowave-sensor power control signal, a video-camera power controlsignal, a microphone power control signal, and a communication-unitpower control signal as required, and supplies the generated signals tothe power controller 452, and also generates a communication controlsignal to a communication unit 426. The abnormality detector 451generates an abnormality detection signal, and supplies it to the videocamera 422 and the microphone 423.

Based on the microwave-sensor power control signal, the video-camerapower control signal, the microphone power control signal, and thecommunication-unit power control signal supplied from the abnormalitydetector 451, the power controller 452 controls the power supply unit427 to supply power to the microwave sensor 425, the video camera 422,the microphone 423, and the communication unit 427 from the batteries431.

Object detection processing performed by the security camera unit 411shown in FIG. 20 is described below with reference to the flowchart ofFIG. 23.

The infrared sensor 424, which is operable by being constantly suppliedwith power from the batteries 431 via the power supply unit 427,receives infrared rays emitted from an object including a human (notshown), and outputs a corresponding detection signal.

In step S201, the abnormality detector 451 reads the detection signal.In step S202, the abnormality detector 451 determines whether anabnormality has occurred based on the detection signal supplied from theinfrared sensor 424. More specifically, the abnormality detector 451determines whether the output level of the detection signal from theinfrared sensor 451 is greater than or equal to a predeterminedreference value. If it is determined in step S202 that the output levelof the detection signal is greater than or equal to the predeterminedreference value, the abnormality detector 451 determines that anabnormality has occurred. If it is determined that the output level ofthe detection signal is smaller than the predetermined reference value,the abnormality detector 451 determines that an abnormality has notoccurred.

For example, as shown in FIG. 24, the abnormality detector 451determines whether the output level of a detection signal 531 from theinfrared sensor 424 is greater than or equal to a reference value T₁.

If it is determined in step S202 that an abnormality has not occurred,i.e., the output level of the detection signal from the infrared signal424 is smaller than the reference value T₁, the process returns to stepS201, and the security camera unit 411 enters the standby mode until theinfrared sensor 424 detects an object. Power is not supplied to themicrowave sensor 425, the microphone 423, the video camera 422, and thecommunication unit 426 until the infrared sensor 424 detects an object.Accordingly, the batteries 431 can be prevented from wastefullyconsuming power, and thus, they can be used over a long period of time.

If it is determined in step S202 that an abnormality has occurred, theabnormality detector 451 generates a microwave-sensor power controlsignal and supplies it to the power controller 452. Upon receiving amicrowave-signal power control signal from the abnormality detector 451,in step S203, the power controller 452 controls the power supply unit427 to supply power of the batteries 431 to the microwave sensor 425,thereby allowing the microwave sensor 425 to be operable.

The microwave sensor 425 generates microwaves and detects reflectionwaves generated when the microwaves are reflected by an object. Themicrowave sensor 425 then generates two detection signals indicatingwhether the object is approaching the microwave sensor 425 and whetherthe object is going away from the microwave sensor 425 based on phasechanges, and supplies the two signals to the abnormality detector 451.In step S204, the abnormality detector 451 reads the two detectionsignals from the microwave sensor 425. In step S205, the abnormalitydetector 451 determines whether an abnormality has occurred based on thetwo detection signals.

More specifically, the abnormality detector 451 determines whether theoutput level of the detection signal indicating that the object isapproaching the microwave sensor 425 and the output level of thedetection signal indicating that the object is going away from themicrowave sensor 425 are greater than or equal to a predeterminedreference value. If the output level of the detection signal indicatingthat the object is approaching is found to be greater than or equal tothe predetermined reference value, the abnormality detector 451determines that an abnormality has occurred. If the output level of thedetection signal indicating that an object is going away from theinfrared sensor 424 is found to be greater than or equal to thepredetermined reference value, the abnormality detector 451 determinesthat an abnormality has not occurred. If the output level of thedetection signal indicating that an object is approaching is smallerthan the predetermined reference value, or if the output level of thedetection signal indicating that an object is going away from theinfrared sensor 424 is smaller than the predetermined reference value,the abnormal detector 451 determines that an abnormality has notoccurred.

Thus, even if an object has been detected by the infrared sensor 424, itis determined that an abnormality has not occurred if the object isgoing away from the infrared sensor 424 or is still. That is, theprocess returns to step S201, and step S201 and the subsequent steps arethen repeated. Accordingly, power consumption can be inhibited comparedto the case in which the video camera 422 or the microphone 423, whichconsumes large power, is immediately operated upon detecting an objectby the infrared sensor 424.

For example, since the output level of the detection signal 531 from theinfrared sensor 424 is smaller than the reference value T₁ before timet₁, the microwave sensor 425 is not operated. In contrast, at time t₁,the output level of the detection signal 531 supplied from the infraredsensor 424 reaches the predetermined reference value T₁. Then, theabnormality detector 451 determines that an abnormality has occurred,and generates a microwave-sensor power control signal and supplies it tothe power controller 452. The power controller 452 controls the powersupply unit 427 to supply power of the batteries 431 to the microwavesensor 425 based on the microwave-sensor power control signal suppliedfrom the abnormality detector 451. Then, the power source of themicrowave sensor 425 is switched from OFF to ON, and the microwavesensor 425 starts generating microwaves.

The microwave sensor 425 detects reflection waves generated whenmicrowaves are reflected by an object so as to generate a detectionsignal 541 indicating that an object is going away from the microwavesensor 425 and a detection signal 551 indicating that an object isapproaching the microwave sensor 425, and supplies the detection signals541 and 551 to the abnormality detector 451. In the example shown inFIG. 24, at time t₂, the microwave sensor 245 outputs the detectionsignal 551 indicating that an object is approaching. The abnormalitydetector 451 determines that an object is approaching the microwavesensor 425 when the output level of the detection signal 551 is greaterthan or equal to a predetermined reference value T₂.

As stated above, the detection signal 541 is a signal indicating that anobject is going away from the microwave sensor 425. Since an object isapproaching the microwave sensor 425, the output level of the detectionsignal 541 remains 0 In the example shown in FIG. 25, at time t₁₁, theoutput level of the detection signal 531 from the infrared sensor 424reaches the predetermined reference value T₁, and thus, power issupplied to the microwave sensor 425. At time t₁₂, the detection signal541 indicating that an object is going away from the microwave sensor425 is output. The abnormality detector 451 determines that noabnormality has occurred when the output level of the detection signal541 is greater than or equal to a reference value T₃.

In the example shown in FIG. 25, since an object is going away from themicrowave sensor 425, the output level of the detection signal 551indicating that an object is approaching the microwave sensor 425remains 0.

As described above, after detecting an object by the infrared sensor424, the microwave sensor 425 further determines whether the object isapproaching the security camera unit 411. Thus, abnormalities can bedetected more precisely, for example, even if an object has beendetected, it can be determined that no abnormality has occurred if theobject is going away from the security camera unit 411.

If it is determined in step S205 that an abnormality has occurred (anobject is approaching), the process proceeds to step S206. In step S206,the abnormality detector 451 generates a microwave-sensor power controlsignal, a video-camera power control signal, a microphone power controlsignal, and a communication-unit power control signal, and supplies themto the power controller 452.

In this embodiment, both the power sources of the video camera 422 andthe microphone 423 are turned ON. However, only the power source of thevideo camera 422 or only the power source of the microphone 423 may beturned ON.

In step S206, based on the microwave-sensor power control signal, thevideo-camera power control signal, the microphone power control signal,and the communication-unit power control signal supplied from theabnormality detector 451, the power controller 452 controls the powersupply unit 427 to supply power of the batteries 431 to the video camera422, the microphone 423, and the communication unit 426, and also tostop supplying power to the microwave sensor 425. Then, the powersources of the video camera 422, the microphone 423, and thecommunication unit 426 are switched from OFF to ON, and the power sourceof the microwave sensor 425 is changed from ON to OFF. The video camera422 is then able to capture images of objects, and the microphone 423 isable to collect sound of and near an intruder outdoors, and the imagesand sound are converted into electric signals. The communication unit426 is able to send outputs of the video camera 422 and the microphone423.

In this manner, power is supplied to the video camera 422 and themicrophone 423, which consume large power, only when the microwavesensor 425 detects that an object is approaching, thereby decreasingpower consumption in the security camera unit 411.

In step S207, the video camera 422 captures an image of an object in thephotographic area, and supplies a corresponding image signal to thecommunication unit 426. The microphone 423 also collects sound in thesound collecting area, converts it into an electric signal, and suppliesa resulting audio signal to the communication unit 426. Thecommunication unit 426 then sends the image signal and the audio signalsupplied from the video camera 422 and the microphone 423, respectively,to the receiver 412 based on the communication control signal suppliedfrom the abnormality detector 451.

The infrared sensor 424, which is operable by constantly being suppliedwith power from the batteries 431 via the power supply unit 427, furtherreceives light from an object including a human (not shown), and outputsa corresponding detection signal. In step S208, the abnormality detector451 reads the detection signal. In step S209, the abnormality detector451 determines whether an abnormality has been detected based on thedetection signal from the infrared sensor 424. That is, the abnormalitydetector 451 determines whether the output level of the detection signalis greater than or equal to the predetermined reference value, as instep S202.

Thus, after transmitting the image signal and the audio signal from thecommunication unit 426, the infrared sensor 424 further detects whetheran object is present. Accordingly, it is possible to more preciselydetect abnormalities, and also, the user is able to easily determinewhether the situation is abnormal.

If it is determined in step S209 that an abnormality has been detected,i.e., the output level of the detection signal from the infrared sensor424 is greater than or equal to the predetermined reference value, theprocess returns to step S207, and steps S207 and S208 are then repeated.Accordingly, processing similar to the above-described processing isrepeated until abnormalities are eliminated, and thus, the user is ableto easily determine whether abnormalities have been eliminated.

If it is determined in step S209 that no abnormality has occurred, i.e.,the output level of the detection signal from the infrared sensor 424 issmaller than the predetermined reference value, the abnormality detector451 generates a video-camera power control signal, a microphone powercontrol signal, and a communication-unit power control signal, andsupplies them to the power controller 452. In step S210, the powercontroller 452 controls the power supply unit 427 to stop supplyingpower to the video camera 422, the microphone 423, and the communicationunit 426 based on the video-camera power control signal, the microphonepower control signal, and the communication-unit power control signal,respectively, supplied from the abnormality detector 451. Accordingly,the power sources of the video camera 422, the microphone 423, and thecommunication unit 426 are changed from ON to OFF. In this manner, powercan be supplied to the video camera 422, the microphone 423, and thecommunication unit 426 only when necessary, thereby enhancing lowerpower consumption.

In this embodiment, three sensors, i.e., the infrared sensor 424, themicrowave sensor 425, and at least one of the video camera 422 and themicrophone 423 (which can be considered to be types of sensors) are usedfor detecting objects. However, other types of sensors may be used.

In this embodiment, the power sources are progressively switched fromOFF to ON in order of increasing power consumption, i.e., in the orderof the infrared sensor 424, the microwave sensor 425, the microphone423, and the video camera 422. If other types of sensors are used, asdescribed above, the power sources are switched from OFF to ON in orderof increasing power consumption.

According to the above-described embodiment, abnormalities can bedetected, and more particularly, abnormalities can be detected moreprecisely while inhibiting power consumption so that batteries can beused over a long period of time.

An embodiment in which an image-capturing device, such as a securitycamera, is installed in a vehicle, is described below.

Reference is first made to FIG. 26. As shown in FIG. 26, a driverreverses a vehicle 601 into a garage, which is surrounded by a housewall 621 and a wall 622, toward the wall 622 in parallel with the housewall 621. An image-capturing device 611 is installed in the back of thevehicle 601 so that the image-capturing direction faces a desireddirection. An image captured by the image-capturing device 611 is sentto a reception display device 612 disposed at the front of the vehicle601 and at the left side (passenger seat) of a steering wheel 602, andis displayed on the reception display device 612. The driver is thenable to understand the distance to the wall 622 or identify an obstaclebetween the vehicle 601 and the wall 622 by checking the image displayedon the reception display device 612 without turning back. This enablesthe driver to park the vehicle 601 in the garage safely and speedily.Another reception display device 613 is disposed indoors near a door 623(outside the vehicle 601) at the house wall 621.

An example of the installment of the image-capturing device 611 isdiscussed below with reference to FIG. 27.

The image-capturing device 611 is disposed on a backseat 631 of thevehicle 601 from which images outside can be captured through a rearwindow 632. The image-capturing device 611 is formed of a multi-sensorcamera unit 641 and a pan tilter 642, and the multi-sensor camera unit641 is rotatably loaded on the pan tilter 642 installed on the backseat631.

The electrical configuration of the image-capturing device 611 isdescribed below with reference to FIG. 28.

A trigger sensor 682 is formed of, for example, a microwave sensor, anddetects the presence or the absence of humans (objects), the distance,and the moving direction. A detection signal obtained by the triggersensor 682 is sent to a microcomputer 676. Generally, glass does nottransmit infrared rays of infrared sensors. Accordingly, infraredsensors cannot detect the motion through the windows of the vehicle 601,and thus, they are not used in this example. However, the trigger sensor682 does not restricted to a microwave sensor, and may be another typeof sensor, for example, a thermal sensor, a vibration sensor, or anacoustic sensor.

An input unit 681 includes a switch, a button, etc. (not shown). Byoperating the input unit 681, the user is able to store a threshold fordetermining the occurrence of an abnormality from a detection signalsent from the trigger sensor 682 in a memory 677 built in themicrocomputer 676.

Upon receiving a detection signal from the trigger sensor 682, themicrocomputer 676 compares the detection signal with the thresholdstored in the memory 677. When the detection signal is found to begreater than or equal to the threshold, the microcomputer 676 controls apower control relay 675 to supply power to a video camera 671, aprocessor 672, and a wireless communication unit 673 from a selector679, thereby operating these elements. When the detection signal issmaller than the threshold, the video camera 671, the processor 672, andthe wireless communication unit 673 are not operated so that powerconsumption can be suppressed. In particular, the video camera 671,which consumes relatively large power, is not operated, which iseffective in suppressing power consumption.

A video signal obtained by the video camera 671 is supplied to theprocessor 672, and is combined with distance information and batteryremaining-amount information supplied from the microcomputer 676 asrequired. The video signal combined with the distance information andthe battery remaining-amount information is further subjected to varioustypes of signal processing in the processor 672, and is then supplied tothe wireless communication unit 673. The video signal is then wirelesslytransmitted from the wireless communication unit 673 to the receptiondisplay devices 612 and 613 via an antenna 674.

The distance information includes, for example, a message correspondingto the distance to an obstacle, for example, “about one more meter”, andis stored in the memory 677.

A battery unit 678 disposed in the multi-sensor camera unit 611 or avehicle battery unit 702 disposed in the vehicle 601 supplies power tothe video camera 671, the processor 672, the wireless communication unit673, the trigger sensor 682, the microcomputer 676, and a driving unit680. Normally, power is supplied from the battery unit 678, and when theremaining amount of the battery unit 678 is smaller than a firstreference value, power is supplied from the vehicle battery unit 702.

The battery unit 678 and the vehicle battery unit 702 are switched bythe selector 679. The microcomputer 676 determines whether the remainingamount of the battery unit 678 is smaller than the first reference valuestored in the built-in memory 677. When the remaining amount of thebattery unit 678 is smaller than the first reference value, themicrocomputer 676 controls the selector 679 to select the vehiclebattery unit 702 and controls the vehicle battery unit 702 to supplypower to the individual elements.

The microcomputer 676 also determines whether the remaining amount ofthe vehicle battery unit 702 is smaller than a second reference value(which is smaller than the first reference value) stored in the memory677. When the remaining amount of the vehicle battery unit 702 is foundto be smaller than the second reference value, the microcomputer 676outputs battery remaining-amount information to the processor 672. Thebattery remaining-amount information is combined with the video signalfrom the video camera 671 as required, and is sent to the receptiondisplay devices 612 and 613. Accordingly, the user is able to know thatthe remaining amount of the batteries becomes small.

A status detector 701 disposed in the vehicle 601 detects the operationstatus of the vehicle 601, and outputs a status detection signalcorresponding to the detection result to the microcomputer 676. Themicrocomputer 676 controls the drive unit 680 to rotate the pan tilter642 at a predetermined position based on the input status detectionsignal. The position at which the pan tilter 642 rotates can beprestored in the memory 677 by the operating the input unit 681 by theuser.

A removable medium 691 formed of a magnetic disk, an optical disc, amagneto-optical disk, or a semiconductor memory is loaded in themicrocomputer 676 as required.

FIG. 29 illustrates the electrical configuration of the receptiondisplay device 612.

A wireless communication unit 801 receives a video image obtained by thevideo camera 671, the distance information obtained by the triggersensor 682, and the battery remaining-amount information from thewireless communication unit 673 of the image-capturing device 611 via anantenna 802.

The video signal, the distance information, the battery remaining-amountinformation received by the wireless communication unit 801 are suppliedto a processor 803. The processor 803 separates the video signal fromthe input signal, and outputs the video signal to a display device 804,for example, an LCD device. The display device 804 then displays animage corresponding to the video signal. The processor 803 alsoseparates the battery remaining-amount information from the inputsignal, and supplies it to the display device 804. The display device804 then displays the battery remaining-amount information, for example,a battery run-out message. The processor 803 also separates the distanceinformation from the input signal, and supplies it to a speaker 805.Then, an audio message, for example, “about one more meter”, is outputfrom the speaker 805. The sound to be output is not restricted to theabove-described example.

Communication between the image-capturing device 611 and the receptiondisplay device 612 are wirelessly performed. However, since bothelements are disposed in the vehicle 601, communication therebetween maybe performed by wired means. The display device of a car navigationsystem or a car television receiver may also be used as the displaydevice 804. Alternatively, the reception display device 612 may also beused as a car navigation system or a car television receiver.

The electrical configuration of the reception display device 613 isdiscussed below with reference to FIG. 30.

The video signal and the battery remaining-amount information receivedby a wireless communication unit 901 via an antenna 902 are supplied toa processor 903. The processor 903 separates the video signal from theinput signal, and supplies it to a display device 904. The displaydevice 904 then displays an image corresponding to the video signal. Theprocessor 903 separates the battery remaining-amount information formthe input signal, and outputs it to the display device 904. The displaydevice 904 then displays the battery remaining-amount information, forexample, a battery run-out message.

The processor 903 also sends the received video signal and the batteryremaining-amount information to a user's mobile terminal via acommunication unit 905. Accordingly, the user is able to view thereceived video signal and to recognize that the batteries are runningout even if the user is not near the reception display device 613disposed indoors.

Since the image-capturing device 611 is disposed inside the vehicle 601,and the reception display device 613 is disposed indoors (outside thevehicle 601), communication between the two elements is performed bywireless means. The display device of a television receiver or aninterphone disposed indoors may also be used as the display device 904.

The image-capturing device 611 is used not only for a back monitor forreversing the vehicle 601, but also for a monitor for households.Accordingly, the user is required to register the monitoring directionin advance before starting to use the image-capturing device 611.

A description is now given of pre-registration processing for themonitoring direction with reference to the flowchart of FIG. 31. Thisprocessing is performed while the user parks the vehicle 601 at a normalposition in the garage.

The user inputs an instruction to set the monitoring direction throughthe input unit 681. Then, in step S301, the microcomputer 676 turns ONthe power control relay 675 to supply power output from the battery unit678 via the selector 679 to the video camera 671, the processor 672, andthe wireless communication unit 673, thereby operating these elements.

The user then inputs a moving position by operating the input unit 681so that the video camera 671 faces the monitoring direction. In stepS302, the microcomputer 676 reads the designated position of the pantilter 642.

In step S303, the microcomputer 676 controls the drive unit 680 torotate the pan tilter 642 at the position read in step S302. In theexample shown in FIG. 32, the pan tilter 642 is rotated so that thephotographic direction of the video camera 671 is set so that the videocamera 671 captures images of the house wall 621.

In step S304, the microcomputer 676 controls the wireless communicationunit 673 to transmit the video signal obtained (captured) by the videocamera 671 to the reception display device 612. The processor 803 of thereception display device 612 outputs the video signal received by thewireless communication unit 801 via the antenna 802 to the displaydevice 804, and the video signal is displayed on the display device 804.Since the video signal is transmitted to the reception display device612 installed in the vehicle 601, the user is able to immediately checkthe image displayed on the display device 804 of the reception displaydevice 612.

The user continues to operate the input unit 681 until the image in adesired monitoring direction is displayed by checking the screen of thedisplay device 804. When the image in the desired monitoring directionis displayed, the user inputs an instruction “OK” by operating the inputunit 681. Then, in step S305, the microcomputer 676 determines whetheran instruction “OK” is input through the input unit 681.

If it is found in step S305 that an instruction “OK” is not input by theuser through the input unit 681, i.e., if the image displayed on thereception display device 612 does not coincide with the image within themonitoring area desired by the user in step S304, the process returns tostep S302, and the position of the pan tilter 642 is moved.

For example, when the video camera 671 captures an image of the door623, as shown in FIG. 32, the user operates the input unit 681 to inputan instruction “OK”.

If it is determined in step S305 that an instruction “OK” is input bythe user through the input unit 681, i.e., if the image displayed on thereception display device 612 substantially coincides with the image inthe monitoring area desired by the user in step S304, the processproceeds to step S306. In step S306, the microcomputer 676 stores theposition of the pan tilter 642 in the built-in memory 677.

In step S307, the microcomputer 676 turns OFF the power control relay675 to stop supplying power to the video camera 671, the processor 672,and the wireless communication unit 673, thereby rendering theseelements inoperable. Then, the monitoring-direction pre-registrationprocessing is completed. This processing is performed only once unlessthe monitoring direction is changed.

Sensor processing performed by the microcomputer 676 is described belowwith reference to FIG. 33.

In step S310, the microcomputer 676 receives a status detection signalfrom the status detector 701 disposed in the vehicle 601, and determineswhether the engine key of the vehicle 601 has been removed based on thereceived status detection signal.

If it is determined in step S310 that the engine key has been removed,i.e., when the vehicle 601 is parked, in step S313, the microcomputer676 performs monitor mode processing. Details of the monitor modeprocessing are given below with reference to FIG. 35. In this manner,when the engine key is removed, the vehicle 601 automatically enters themonitor mode without requiring a special operation for setting themonitor mode. As a result, the situation in which the user forgets toset the monitor mode can be prevented, and monitoring can be reliablyperformed.

If it is determined in step S310 that the engine key has not beenremoved, the process proceeds to step S311. In step S311, themicrocomputer 676 determines whether the transmission gear is at theback position (the position at which the vehicle 601 is reversed) basedon the status detection signal received in step S310.

If it is found in step S311 that the transmission gear is at the backposition, in step S312, the microcomputer 676 performs back monitorprocessing. Details of the back monitor processing are given below withreference to FIG. 34.

If it is found in step S311 that the transmission gear is not at theback position, that is, when the vehicle 601 is still or advances, orafter step S312 or S313, the process returns to step S310, and step S310and the subsequent steps are then repeated.

Back monitor processing is described below with reference to FIG. 34.

Back monitor processing is performed when the transmission gear is atthe back position, namely, when the vehicle 601 is reversed, forexample, into the garage.

In step S315, the microcomputer 676 controls the drive unit 680 to drivethe pan tilter 642 so that the video camera 671 can capture images atthe back of the vehicle 601 through the window 632.

In step S316, the microcomputer 676 sets the communicating party to bethe reception display device 612 disposed in the vehicle 601 rather thanthe reception display device 613 disposed outside the vehicle 601.

In step S317, the microcomputer 676 turns ON the power control relay675, and also controls the selector 679 to select power from the batteryunit 678 so as to supply power to the video camera unit 671, theprocessor 672, and the wireless communication unit 673, therebyoperating these elements.

If the settings in steps S315, S316, and S317 have already been madebefore the processing, these steps are ignored.

In step S318, the microcomputer 676 transmits a video signal obtained bythe video camera 671 and subjected to various types of signal processingby the processor 672 to the reception display device 612 via the antenna674 from the wireless communication unit 673.

In step S319, the microcomputer 676 reads distance information obtainedby the distance sensor of the trigger sensor 682.

In step S320, the microcomputer 676 reads an audio message correspondingto the distance information read in step S319 from the memory 677, andsends the audio message to the reception display device 612.

The above-described processing is repeatedly performed in the route ofsteps S310, S311, and S312 of FIG. 33. The processor 803 of thereception display device 612 processes the signal received by thewireless communication unit 801 from the wireless communication unit 673via the antenna 802, and outputs the video signal to the display device804 and outputs the audio signal as the distance information to thespeaker 805. Accordingly, an image in the back of the vehicle 801 isdisplayed on the display device 804, and an alarm message correspondingto the distance information, for example, “three more meters”, “two moremeters”, or “one more meter”, is issued from the speaker 805. The user(driver) is thus able to reverse the vehicle 601 safely and comfortablyby checking the display device 804 in front without turning back.

The audio message corresponding to the distance information may bestored in a storage unit (not shown) of the reception display device 612rather than storing in the memory 677 of the microcomputer 676, and whenthe reception display device 612 receives the distance information, theaudio message may be read from the storage unit.

Details of the monitor mode processing in step S313 of FIG. 33 aredescribed below with reference to FIG. 35.

As discussed above, this processing is automatically performed when theengine key of the vehicle 601 has been removed (when the vehicle 601 isparked).

In step S321, the microcomputer 676 sets the communicating party to bethe reception display device 613 disposed indoors rather than thereception display device 612 inside the vehicle 601.

In step S322, the microcomputer 676 drives the drive unit 680 to rotatethe pan tilter 642 at the position stored in the memory 677 (positionstored in step S306 of FIG. 31). The monitor direction in the monitormode may be set by operating the input unit 681 by the user. In thiscase, however, the user is required to set the direction, for example,every time the user parks the vehicle 601. Accordingly, if the positionof the pan tilter 642 is registered in advance, the direction isautomatically set by parking the vehicle 601 at the predeterminedposition (by removing the engine key). Thus, the ease of operation canbe enhanced, and also, the situation in which the user forgets to setthe direction can be eliminated, thereby allowing the monitoringprocessing to be reliably performed. In this manner, the monitored areaset by the user (an area other than the vehicle 601) is monitored by thevideo camera 671 provided for the vehicle 601, thereby decreasing thepossibility an intruder realizing that he/she is monitored, and alsomaking it possible to reliably monitor an intruder at a very neardistance of the monitored area. As a result, detailed information of anintruder can be obtained and reported to the user.

Additionally, since the video camera 671 is disposed in the vehicle 601,the possibility of an intruder realizing that he/she is monitored issmaller compared to the case in which the video camera 671 is disposedoutside the vehicle 601, for example, on the roof the vehicle 601.Accordingly, the video camera 671 is hidden inside the vehicle 601,thereby preventing monitoring to be interfered with. This is veryeffective in monitoring not only households, but also, for example,stores.

In step S323, the microcomputer 676 turns OFF the power control relay675 to stop supplying power to the video camera 671, the processor 672,and the wireless communication unit 673, thereby rendering theseelements inoperable.

If settings in steps S321, S322, and S323 have already been made beforethe processing, these steps are ignored.

In step S324, the microcomputer 676 reads the distance informationobtained by the distance sensor of the trigger sensor 682.

In step S325, the microcomputer 676 determines whether there is a changein the distance information read in step S324.

If it is found in step S325 that there is a change in the distanceinformation, the process proceeds to step S326. In step S326, themicrocomputer 676 turns ON the power control relay 675 to supply powerto the video camera unit 671, the processor 672, and the wirelesscommunication unit 673 from the battery unit 678, thereby operatingthese elements. Accordingly, the video camera unit 671, the processor672, and the wireless communication unit 673 are not operated undernormal conditions, and only when there is a change in the distanceinformation, these elements are operated, thereby making it possible tosuppress power consumption of the video camera unit 671, the processor672, and the wireless communication unit 673, which consume relativelylarge power.

In step S327, the microcomputer 676 sends the video signal obtained bythe video camera 671 and subjected to various types of signal processingby the processor 672 to the reception display device 613 via thewireless communication unit 673.

In step S328, the microcomputer 676 controls the drive unit 680 torotate the pan tilter 642 in the direction of a moving object.Accordingly, the video camera 671 can constantly transmit the videosignal of the moving object to the reception display device 613.

In step S329, the microcomputer 676 sends an alarm (for example, acombination of characters and sound “abnormal condition has beendetected”) to the reception display device 613 via the wirelesscommunication unit 673.

If it is found in step S325 that there is no change in the read distanceinformation, in step S330, the microcomputer 676 detects the remainingamount of the battery unit 678.

In step S331, the microcomputer 676 determines whether the remainingamount of the battery unit 678 is smaller than the first reference valuestored in the built-in memory 677.

If the outcome of step S331 is yes, in step S332, the microcomputer 676controls the selector 679 to select the power of the vehicle batteryunit 702, thereby preventing the discontinuous supply of power andmaking it possible to perform monitoring over a long period of time.

If it is found in step S331 that the remaining amount of the batteryunit 678 is greater than or equal to the first reference value, or afterstep S332, the process proceeds to step S333. In step S333, themicrocomputer 676 determines whether the remaining amount of the vehiclebattery unit 702 is smaller than the second reference value stored inthe memory 677.

If the outcome of step S333 is yes, in step S334, the microcomputer 676sends the battery remaining-amount information to the reception displaydevice 613. Accordingly, the user can be always informed that thebatteries are running out. This processing is effective not only inreporting the remaining amount of the battery unit 678 of the videocamera 671, but also in reporting the remaining amount of the vehiclebattery unit 702.

After step S329 or S334, or if it is found in step S333 that theremaining amount of the vehicle battery unit 702 is greater than orequal to the second reference value, the process returns to step S310 ofFIG. 33, and step S310 and the subsequent steps are then repeated.

The above-described series of processings may be executed by hardware orsoftware. If software is used, the processings are executed by acomputer in which a corresponding software program is integrated intodedicated hardware. Alternatively, the corresponding software program isinstalled from a recording medium into a computer, for example, ageneral-purpose computer that is able to execute various functions byinstalling various programs. An example of the general-purpose computeris the personal computer 500 shown in FIG. 19.

The above-described recording medium includes not only a package mediumformed of the removable medium 691, such as a magnetic disk, an opticaldisc, a magneto-optical disk, or a semiconductor memory, recording theprogram therein, as shown in FIG. 28, which is distributed to the userseparately from the multi-sensor camera unit 641, but also the memory677 recording the program therein, which is provided to the user bybeing integrated into the multi-sensor camera unit 641.

Steps of the computer program recorded in the recording medium may beperformed in chronological order described in this embodiment, and alsomay be performed concurrently or individually.

According to this embodiment, vehicles can be safely reversed.Additionally, it is possible to monitor outside the vehicle when it isparked without making an intruder realize that he/she is monitored.

What is claimed is:
 1. A system comprising: circuitry of a vehicleconfigured to detect a remaining amount of a battery of the vehicle in anon-charging state of the battery of the vehicle, determine whether thedetected remaining amount of the battery of the vehicle in thenon-charging state of the battery is less than a predetermined referencevalue, the predetermined reference value being less than a battery fullamount value, supply battery remaining-amount information regarding theremaining amount of the battery of the vehicle in the non-charging stateof the battery on condition that the remaining amount of the battery ofthe vehicle in the non-charging state of the battery is less than thepredetermined reference value, switch communication receiving partiesfrom a first communication terminal inside the vehicle to a secondcommunication terminal external to the vehicle at a first location, andcause wireless transmission of the battery remaining-amount informationregarding the vehicle in the non-charging state of the battery to thesecond communication terminal external to the vehicle at the firstlocation, the second communication terminal being configured towirelessly transmit the battery remaining-amount information regardingthe vehicle in the non-charging state of the battery to a portablemobile communication terminal at a second location different from thefirst location.
 2. The system according to claim 1, further comprisingwireless communication circuitry that wirelessly transmits the batteryremaining-amount information to the portable mobile communicationterminal.
 3. The system according to claim 1, further comprising: asteering wheel to control the vehicle.
 4. The system according to claim1, further comprising drive circuitry that drives a rotatable apparatusattached to the vehicle.
 5. The system according to claim 1, wherein thebattery remaining-amount information is battery run-out information. 6.The system according to claim 1, wherein the battery remaining-amountinformation is part of a wirelessly transmitted signal that includes avideo signal, which is obtained by a video camera attached to thevehicle.
 7. The system according to claim 6, wherein the wirelesslytransmitted signal includes the video signal combined with the batteryremaining-amount information and distance information regarding adistance of the vehicle from an object as detected from an imagecaptured by the video camera.
 8. The system according to claim 1,wherein the circuitry is configured to abstain from causing transmissionof the battery remaining-amount information regarding the vehicle in thenon-charging state of the battery when the battery remaining-amount isgreater than or equal to the predetermined reference value.
 9. A methodcomprising: detecting a remaining amount of a battery of a vehicle as anamount of the battery is decreasing, determining whether the detectedremaining amount of the battery of the vehicle as the amount of thebattery is decreasing is less than a predetermined reference value, thepredetermined reference value being less than a battery full amountvalue, outputting, using a processor, battery remaining-amountinformation regarding the detected remaining amount of the battery ofthe vehicle responsive to the detected remaining amount of the batteryof the vehicle being less than the predetermined reference value,switching communication receiving parties from a first communicationterminal inside the vehicle to a second communication terminal externalto the vehicle at a first location, and wirelessly transmitting thebattery remaining-amount information to the first communication terminalexternal to the vehicle at the first location, the first communicationterminal being configured to wirelessly transmit the batteryremaining-amount information to a portable mobile communication terminalat a second location different from the first location.
 10. The methodaccording to claim 9, wherein the battery remaining-amount informationis battery run-out information.
 11. The method according to claim 9,wherein the battery remaining-amount information is part of a signalthat includes a video component obtained by a video camera attached tothe vehicle, the video component being combined with the batteryremaining-amount information.
 12. The method according to claim 9,further comprising abstaining from performing said outputting thebattery remaining-amount information regarding the battery of thevehicle as the amount of the battery is decreasing responsive to saidcomparing indicating that the detected remaining amount of the batteryof the vehicle as the amount of the battery is decreasing is greaterthan or equal to the predetermined reference value.
 13. The methodaccording to claim 9, wherein the battery remaining-amount informationis wirelessly transmitted to the portable mobile communication terminal,which is comprised of circuitry configured to receive wirelessly thebattery remaining-amount information regarding the battery of thevehicle as the amount of the battery is decreasing, and to display thebattery remaining-amount information on a display.
 14. The systemaccording to claim 1, wherein the portable mobile communication terminalis a wireless remote control apparatus.
 15. The method according toclaim 9, wherein the portable mobile communication terminal is awireless remote controller.
 16. The system according to claim 1, whereinthe circuitry is configured to cause wireless transmission of thebattery remaining-amount information as part of a signal to the portablemobile communication terminal, and to also cause wireless transmissionof the signal including the battery remaining-amount information of thevehicle to the first communication terminal inside the vehicleconfigured to display the battery remaining-amount information.
 17. Thesystem according to claim 1, further comprising wirelessly transmittingthe battery remaining-amount information of the vehicle to the firstcommunication terminal inside the vehicle configured to cause display ofthe battery remaining-amount information.
 18. The system according toclaim 1, wherein the circuitry is configured to switch communicationreceiving parties from the second communication terminal external to thevehicle at the first location to the first communication terminal insidethe vehicle.
 19. The method according to claim 9, wherein the switchingcommunication receiving parties from the first communication terminalinside the vehicle to the second communication terminal external to thevehicle at the first location is responsive to detection of an enginekey of the vehicle having been removed.